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... PREFACE 

A comprehensive multispectral program devoted to the advancement of 
state-of-the-art techniques for remote sensing of the environment has 
been a continuing program at the Environmental Research Institute of 
Michigan (ERIM), formerly the Willow Run Laboratories of The University 
of Michigan. The basic objective of this multidisciplinary program is 
to develop remote sensing as a practical tool to provide the user with 
processed information quickly and economically. 

The importance of providing timely information obtained by remote 
sensing to such people as the farmer, the conservationist, and others 
concerned with problems such as crop yield and disease, urban land 
studies and development, water pollution, and forest management must be 
carefully considered in the overall program. The scope of our program 
includes: (1) extending the understanding of basic processes, (2) dis- 

covering new applications, (3) developing advanced remote- sensing 
systems, (4) improving fast automatic data processing systems to extract 
information in a useful form, and also (5) assisting in data collection, 
processing, analysis and ground truth verification. The MIDAS program 
applies directly to No. (4) with its improved data processing capability. 
The follow-on study of on-board processing applies to both items (3) 
and (4 ) . 

The contract effort was monitored by Mr. William M. Howie of NASA-r 
Langley . The over-all program was monitored by Mr. R. R. Legault, a Vice 
President of ERIM and Director of the Infrared and Optics Division. 

Work on this contract was directed by J. D. Erickson, Head of the Infor- 
mation Systems and Analysis Department, and by F. J. Kriegler, Principal 
Investigator. The ERIM number for this report is 108800-54-F. 
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ERIM personnel who contributed to this project and who co-authored 
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1 

INTRODUCTION 


The introduction of modern sensors aboard spacecraft for earth re- 
sources survey permit large portions of the earth to be rapidly remotely 
sensed by these devices. Since these multispectr'al scanners have fine 
spectral and spatial resolution, the amount. of data collected will have 
typical average' data rates from 10 to 100 megabits/sec. A special pur- 
pose processing system called MIDAS* has been developed as an attempt 
to cope with these rates and overcome the bottlenecks encountered with 
processing techniques employing 'general purpose machines . ;However, a 
special purpose ground-based system does not reduce the problems asso- 
ciated with wide band data transmission links or storage of these trans- 
mitted data. Clearly, there is a need for processing scanner data 
aboard a spacecraft as a means of reducing the peak and average loads 
of data to be transmitted- to the ground, the data storage. requirements 
on the ground, and also the number , size, and cost of processing systems 
on the ground. To quote the, "Outlook for Space — A Synopsis": 

,"lt must be emphasized that the satellites . are only one 
part of the overall system. The operational costs include 
development and operation of ah information management sys- 
tem to process the data, provide the actual crop prediction, 
and improve the theoretical models which underlie data inter- 
pretation. The data obtained from this Satellite system 
would be useful for a number of additional objectives included 
in the study which require multi-spectral images at comparable 
resolution; *' 

The. need for improvement in data management and informa- 
tion extraction systems cannot be overemphasized. Consider, 
for example, the magnitude of the task. The surface of the 
Earth consists of about 564,000 billion square meters. The 
satellites contemplated even in the first phase would view 
this surface as if it were divided into small 1000-meter 


* IIIDAS stands for Multivariate interactive Digital Analysis System. 
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squares, or 564,000 million such small squares. Every twenty- 
four hours each satellite would be capable of measuring each 
such little square in seven' spectral ranges. Even if only 
the dry land area were observed, it would still amount to 
seven measurements in each of 200,000 million squares. 

Much of this information is of little use for the pre- 
diction of food crops, since only a fraction of the surface 
of the Earth is devoted. to farming. But how does one tell 
the satellite what area is farming and what is tundra, desert, 
or jungle? Actually, it could be possible to make such data 
decisions automatically with a computer either on-board the 
satellite or at data processing stations on the surface. 

But even with all of the possible "data compression" — the 
process that cuts down the total .amount of information to 
the minimum absolute necessity — the quantity of data is 
still enormous and must be processed frequently. The solu- 
tion of the problem of turning these data into useful infor- 
mation — useful to those who will plan the harvesting, dis- 
tribution, and marketing of the world's food crop — is 
formidable, but obviously vital to the success of the program. 

A similar need exists for many of the systems examined by 
the Study Group. "[1] 

The continued rapid development of semiconductor technology which 
results in creating new families of devices provides the implementation 
basis for newer,' unique processing systems. Both digital and analog 
processing techniques have been successfully used for the processing of 
multispectral data. [2] These techniques use parallel processing elements 
to obtain high speed operation. It appears that new integrated circuit 
techniques will not only permit economical parallel processing techniques 
used previously but will allow and give rise to multiple processors or 
a matrix of processors (parallel-parallel). The intermediate or final 
output of any processor module could be transmitted when requested by a 
specific user of the processed data.. 

A particularly attractive way of mechanizing the required functions 
is with one of the most recent families of semiconductor innovations, 
charge-coupled devices (CCD's). These can, with the inherent low power 
of these devices, perform a large number, of parallel computations more 
effectively than digital devices. However, there does not seem to have 
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been, as yet, systematic developments at performing general arithmetic, 
or at least, of the operations needed to perform the processing needed 
for remotely sensed multispectral data. 

It is the purpose of this report to define some tentative mechaniza- 
tions for the needed processing operations in detailed block form. These 
first-pass designs will serve as a guide for the use of the circuit/system 
designer and enable iterations toward more practical superior circuits. 

These modules can be designed to perform various processing opera- 
tions which serve to reduce and compress the sensed data in various ways. 
Data may be processed to the point of classifying each element in the 
scene in a supervised or an unsupervised mode. Coding techniques may 
then be employed to further improve the efficiency of transmission. 

Lastly, newer forms of combined spatial and spectral clustering may be 
used to reduce the amount of data to be transmitted even further. As a 
result, it seems reasonable to expect that for some forms of data im- 
provements of more than two orders of magnitude may be obtained using 
on-board processing for reducing the amount of data to be transmitted. 
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2 

A GENERAL ON-BOARD PROCESSING SYSTEM 


The general system for on-board processing is shown in block diagram 
form in Figure 1. There are four- basic parts to this system: 

1. The scene to be viewed - ' 

2. The airborne/spaceborne system 

3. The ground receiving station 

4. ‘ The user processing system. 

The scene to be viewed is an important part of the system and must be 
considered first in some detail before considering the remaining part. 

The scene to be viewed may be grossly broken into two parts: 

1. The ground scene 

2. The intervening atmosphere. 1 

In general more is known about the ground scene than the intervening atmos- 
phere for any particular remote sensing application. Spectral characteris- 
tics of many materials have been measured and catalogued. These measure- 
ments have been used as part of an analysis to determine which spectral 
bands are best for discrimination in particular applications .[ 3] However, 
most of these analysis techniques, to date, have not considered the effects 
of the intervening atmosphere which may seriously affect the received ra- 
diation's. spectral characteristics at the sensor. A preliminary study 
[4] shows some of the effects of the atmosphere by translating measured 
spectral; characteristics of materials to apparent radiance at a spacecraft 
sensor. 

Another important system consideration contained in the scene to be 
viewed is the resolution required to discriminate the various objects in 
the scene. Studies have been made [3, 5] which detail these requirements. 
Because of the large variation in the spectral/spatial needs it would be 
desirable for an airborne or spaceborne system to have some degree of flex- 
ibility. However, in place of flexibility the system might be tailored to 
a restricted set of applications. Restricted set of applications now 
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exist, in a loose sense, as a result of NASA's current division of its 
remote sensing operations into regional centers. These regional centers 
maintain aircraft/spacecraft data collection facilities, data processing 
facilities, and a staff of applications personnel able to direct and assess 
the effectiveness of remote sensing techniques. One center, as an example, 
may have a charter for water quality and shoreline erosion monitoring of 
the Eastern Seaboard. When such tasks become part of the overall system 
they, in effect, define the spectral bands, the spectral resolution, and 
also the processing techniques. For this example the nominal spectral 
resolution would be about 20 meters [3, 5, 6] with a minimum of 8 spectral 
bands. Of these bands, four to 8 would be useful at any one time. A 
sensor aboard a spacecraft could perform this task. Assuming 8 bit data 
values and a 50 kilometer swath width, the transmission of all the data 
would require a data link capable of transmitting 80 megabits per second. 

As noted previously, for any given application not all this data may be 
needed. 

The amount of data may be reduced before transmission, by prepro- 
cessing the data or by classification of the data as shown in Figure 2. 

To reduce the data prior to transmission two procedures of information 
extraction are: classification with training (i.e., supervised learning), 

and without training (i.e., unsupervised' learning). In the first case, 
some data' is transmitted to the ground station, identified by the opera- 
tor who then transmits classification paramaters to the spacecraft classi- 
fier, which then transmits the class codes of each element of the sensed 
or stored data. Of particular importance in such processing is the mini- 
mization of operator /interpreter intervention. For example, for a . 10.- bit/ 
sec data link, transmission of 8 channels from a 50 sq.:km. scene would 
require about one minute. For the training mode of operation any time 
spent in analyzing and the transmitting of classification parameter is 
inserted into the transmission loop thereby imposing a severe reduction 
in throughput on the system. This may reduce possible data compressions 
to such low values as to produce a system of marginal utility. Fortun- 
ately, however, there are now several experiments which indicate that 




FIGURE 2. DETAILED BLOCK DIAGRAM OF ONBOARD PROCESSOR FOR REMOTELY SENSED DATA 
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satisfactory results may be had using cluster procedures (unsupervised 
training) which compare well with trained classification procedures. For 
this case, the cluster classifier examines the. ■ incoming data, specifies 
clusters, and transmits the cluster codes for each pixel, leaving identi- 
fication of the clusters for the interpreter to perform. There may be 
several intermediate modes of processing as well which may also prove 
advantageous: adaptive recognition with training, adaptive clustering, and 

partially trained clustering. 

In either processing technique the amount of data compression pos- 
sible is, as a minimum, roughly the same. With each pixel (8 channel 
spectral data, quantized to 8 bits) classification into one of 16 to 64 
classes would be typical. This results into a data compression of . about 
12 to one. This indicates that on-board processing techniques using 
classification processing is a suitable approach for data compression. 


9 
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3 

PRE-PROCESSING 

3.0 PRE-PROCESSING 

Pre-processing, or pre-classification processing (as defined here) 
includes the operations of sensor calibration, scene or radiometric/ 
reflectance calibration, and selection or combination of channels (di- 
mension reduction). Pre-processing also includes the correction of the 
geometric inaccuracies of the scanning process, including control of 
sampling rate to correct non-linear scanning rates and correction and 
matching of spatially arrayed data to that obtained from other scenes 
or to previously specified geodetic references. 

In this context, it is evident that this primary task of matching 
the sensed scene to a classification device is of the greatest impor- 
tance in obtaining output data which is correctly classified or com- 
pressed and which is capable of being put to direct use without 
considerable subsequent processing. Inherent in these operations is 
an increasingly greater understanding of the quantitative behavior of 
the sensible scene or scenes of a wide variety of types under varying 
conditions. Because of its importance, it is desirable to elaborate a 
contextual framework for pre-processing within which the tasks of var- 
ious aircraft or spacecraft missions and of various regional centers 
may be systematically studied and developed. Much of this framework 
has only recently been developed to the point that it now seems possible 
to fit together all the pieces of the problem, which has, heretofore, 
been treated in a rather piecemeal and heuristic manner. 

The following section, then, consists of two parts. A rationale 
for preprocessing, is given consisting of; (1) the spectral structure of 
scenes, and (2) the spatial structure of scenes. 
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3.1 A RATIONALE FOR PRE-PROCESSING 

3.1.1 Spectral Structures of Scenes 

Pre-processing has been a slowly developing and somewhat heuristic 
part of multispectral processing since the start of these programs. It 
appears that a reasonable and systematic foundation may be had for the 
proper transformations and functions applied to this kind of data. As a 
result of the requirements for greater accuracy in data processing,' recent 
works have taken a more systematic look at the kinds of operations involved 
and the implicit, underlying structure of the sensed scene. Such ap- ‘ 5 

proaches are typeified by those of Wheeler, et al [7] and, particularly, 
by Kauth and Thomas [8]. The former approach studies the dimension re- 
duction possible in Landsat data taken at one or four times and concludes 
the the scene, originally contained in four or 16 bands (dimensions), can 
be accurately reproduced in two or 8 dimensions. The analysis is statis- 
tical. Kauth and Thomas, however, present a model for the agricultural 
scene which is closely related to the functional variations exhibited 
by growing vegetations, soils, and haze. This model allows the dynamics 
of the vegetative system as observed in the color space to be exhibited 
and analyzed, and provides a systematic basis for analyzing correction 
functions for haze, crop maturity, and equalization of vegetative reflec- 
tances by ratioing. The model has been called the "Tasseled Cap" and is 
illustrated in Figure 3. 

The "Tasseled Cap" model is a figure formed in color space by the 
temporal trajectories of types of vegetation as they change throughout 
a growing season. Vegetation at the start of the season begins from 
a plane containing distributions of soils and develops spectrally in 
a direction approximately orthogonal 'to this plane, gradually occupy- 
ing a smaller volume, bending over and, finally, forming a diverging 
set of distributions at the end of the season to form the tassel. 

As a result, it becomes evident that, for an agricultural scene, 
if one is interested in soil analysis and mapping, the proper data 
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to transmit or study is that lying in the plane of soils. The proper 
data for vegetative study may, in some cases, be restricted to that 
found in the quasi-planar volume of the vegetative cap. 

Haze has the effect of translating this structure toward a "point" 
of "all haze". Thus, the proper method of correcting this source of 
error may be obtained from the derivation of a haze-measure vector which 
may be used to correct the data functionally or to enable the decision 
to transmit the data or not. 

” The point in the color space about which the ratios of dimensions 
should be formed is the shadow-point. This may be located by the inter- 
section of the bounding volumes of the plane of soils and the slope of 
the peak of the vegetative cap. 

Another point of view may be elaborated to place the "Cap" struc- 
ture in a multitemporal form in which the "Cap" structure may be seen 
to be a projection of the temporal sequence. The new structure may be 
aptly termed the "Caterpillar". This is shown in Figure 4, which 
illustrates the evolution of two crop spectra at the extreme "Cap" 
trajectories. Each of the two crop distributions forms a "fuzzy" 
distribution starting within the soil distributions at time t^ and 
forms a "fuzzy" tube over the growing season, sampled at the times of 
overflight, t_. , hence the term "Caterpillar": Such a structure exhib- 
its the temporal character of crop-spectra development and serves to'v/i 
place this data in a mathematical framework of a form in which addi- 
tional analysis and processing become possible. For example, for crops 
planted at various times, the "Caterpillars" will be displaced by At_., 
the incremental differences. When sampled with adequate density, each 
crop spectrum may be functionally displaced to a reference time and the 
set overlaps or variances may be reduced. One may also investigate 
the proper sampling times and intervals for maximal differentiation of 


14 







FORMERLY WILLOW RUN LABORATORIES. THE UNIVERSITY OF MICHIGAN 


crop species and identify the Nyquist rate of corn, for example. Alter- 
natively, given the identity of the "Caterpillars" of all crops in the 
scene, it may be possible to obtain the distributions of planting and 
harvesting dates for all crop fields examined, should this be worth- 
while. Lastly, such a structure provides a basis for investigation of 
the so-called "delta" classifiers, in which the differences in spectra 
over time are used as the data vector for a classification procedure. ; 

It is the purpose of this brief outline to point out the fact that 
more systematic information about signatures of objects in a scene is 
now becoming available and will develop. Signatures are now being repre- 
sented in dimensions of radiance, space and time making these structures 
more easily related to the physical world. Continued development of 
structures such as these are of considerable importance in assuring the 
availability of adequate methods and technology for a spacecraft pro- ■ 
cessor. 

3.1.2 Spatial Structure of Scenes 

A presumption of the previous section is that all multi-temporal anal- 
ysis is preceded by an operation which places successive scenes in spatial 
registry to make available the spectral-temporal data as a well organized 
set of data. This is not a trivial task and, also, clearly not a task to 
be undertaken aboard a spacecraft, because of the extremely large memory 
requirement (multi-orbit, full growing season) .• It should be noted, how- 
ever, that some signature information of a spectral-temporal nature could 
be stored as a means of assessing crop maturity or of improving processing 
accuracy when the analyses outlined above are better developed. 

There are, however, some kinds of spatial processing which may be done 
effectively within the data from a single time slice, i.e., from a single 
view. These are basically two kinds: spectral-spatial processing and 

spatial correction. The former aims at improving classification accuracy 

by associating contiguous array elements with a central element in a proba- 
bilistic manner or aims at improved compression of data by spectral-spatial 
clustering. The latter is intended to improve the geometrical or geodetic 
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accuracy of the transmitted data so that it may be used directly for pur- 
poses of mapping or for registry with other views of the same scene.- 

Spectral-spatial processing for the purposes of compression is treated 
in the subsequent section on studies proposed (Section 5.4) since an 
appreciable amount of development arid test seems necessary to validate and 
prove the techniques. But, briefly, this method preprocesses data by asso- 
ciating the spatial (x-y) coordinates of a picture element with its spec- 
tral vector so that decisions may be made with the spectrally and spati- 
ally clustered data. 

A particular form of such processing has been developed more exteri- 
sively in which closely contiguous elements are associated with a central 
element and treated as an associated decision region. This approach has 
resulted in the various "9-point" rules in which a 3 x 3 array of scene 
elements -is employed to improve recognition accuracy . These techniques 
can be implemented in 3 x 3 array shift-register structures suited to CCD 
technology. Details may be found in Richardson [ 9 ] • The result of such 
preprocessing is an increased dimensionality of information for each scene 
element prior to classification in a cluster or MIDAS classifier. 

Possibly the most useful spatial operation which may be performed 
aboard a spacecraft, however, is that of geometric correction. The 
reduction in .ground station correction required is in proportion to the 
degree of geometric correction possible aboard a spacecraft. Since a 
number of ground-stations will be required to distribute the data into 
areas where it is needed, there is a useful tradeoff to be considered: 
spacecraft cost/complexity versus cost/complexity of a number of ground 
stations . 

There are several logical degrees of spatial correction possible 
aboard a spacecraft. First, a correction may be made for path skew along 
the scan line, such as is inherent in a Landsat orbit. Secondly, data 
may be corrected to fit it to a geodetic grid in an open-loop fashion by 
using orbital and altitude information in a dynamic manner. Finally, 
data may be corrected to fit a geodetic grid in a closed-loop or referential 
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manner, by constraining the. data to, : f.it a-grid of specific and sensed 
ground references. . .. . - . „ 

Depending on the accuracy required by the ground station and the 
spatial processing power desired or feasible in such a station, each of 
these operations may be studied in a trade-off analysis <to ascertain the • 
desirability of performing such corrections prior to transmission.. In 
the first case, the requirement is for a controlled sampling rate to 
match the dynamic errors of non-linear scan rate and to incorporate a 
displacement of data proportional to longitudinal orbit displacement. 

These corrections are readily implemented. 

Data in the second case may be corrected by measuring and/or correct- 
ing the inertial/pointing coordinates of the spacecraft. This may be 
done by running an interpolation window over the raw data and supplying 
the interpolated, corrected data from the spacecraft or to subsequent 
processors and then from the spacecraft, with or without attitude correc- 
tion. The various interpolation procedures now under test seem easily 
mechanized in small array processors. 

The last case requires that the specified ground/geodetic reference 
points be recognized in the spacecraft and be used to construct the two- 
dimensional re-sampling and interpolation procedure. For this purpose, 
spatial pattern search and recognition procedures are required using multi- 
spectral data and stored "shape" signatures. It is not clear that adequate 
techniques exist for these requirements, however, .two approaches appear 
possible. First, two-dimensional correlation procedures may be used, as 
for example, is done by Lillestrand [10] using a certain number of array 
processors to maintain registry for radar images at microsecond/element 
rates. Or, spatial element algebras may be used as developed by Sternberg 
of ERIM to define invariant patterns for recognition. This is mentioned . 
subsequently (Section 5.4.2)., In either case, continued study seems ne- 
cessary to validate and test such methods. 
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4 

CLASSIFICATION 

4.0 CLASSIFICATION 

Classification is the procedure or algorithm by which a given 
picture element or data vector is categorized as belonging to one dis- 
crete class. The classification process can be described by the order 
of the numerical computation used in partitioning the signal space. 

Three forms of classification procedures will be considered, one quadra- 
tic and two linear procedures. From these, two basic hardware implements 
tions result, one linear and one quadratic. 

4.1 QUADRATIC CLASSIFIER 

The quadratic classification procedure is based on the use of the 
discriminants for multivariate Gaussian density functions, 

-0.5 { (X - M.) T 9T 1 (X - M.) + fcn|0 . | + n Jin ■£}, (1) 

for partitioning the color space. The quadratic calculation can be ex- 
pressed in a number of ways. One method is developed and forms the 
basis for the hardware implementation of the MIDAS system [1]. Basically 
the method involves the decomposition of matrix calculation in Eq. (1) 
into a form. 

x T e _1 x = [x t b][b t x]. (2) 

The form shown on the right side in Eq. (2) is simply a number of 

multiplication of the data vector X times the rows of the square matrix B 

The result of this process is a vector Y, and the final quadratic cal- 

T 2 

culation is simply Y Y = EY^. This is implemented in a pipeline type of 
processor in the MIDAS system. About 80% of the classifier uses the same 
general circuit design shown in block diagram form in Figure 5. Mathe- 
matically, the operation performed by this hardware is described by 

N 

y - X w x , (3) 

y=l J 3 
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FIGURE 5. BLOCK DIAGRAM OF THE MATRIX MULTIPLIER CARD 
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where the w 's are statistical coefficients and the x.'s are the 

j J 

components of the data vector X; 

In Charge-Coupled Device technology, the operation indicated 
by Eq. (3) has long been implemented for permanently stored coefficients 
w n - Charge-coupled devices employing fixed weights, w n , have 
been used as the basis for transversal filters having up to 500 weights 
as shown in Figure 6. These have been implemented in a silicon chip 

with the size of the chip being on the order of 200 mils square. The 
power consumption of such a device is about 100 milliwatts [11, 12], 

In contrast, the similar function implemented with conventional MSI/LSI 
TTL logic in the MIDAS system and having 16 weights is contained on a 
7x7 inch circuit board and consumes about 8 watts of power. The one 
advantage of this MIDAS circuit board is that the variable weights are 
stored in random access memories. 

The two circuits described above form the basis for constructing 
a classifier using CCD's. The fundamental computation to be performed 
is 


N 

Y = £ h(nt)x(nt) 

n=l 


N 


L 

n=l 


h 


x 

n n 


(4) 


A representation of this is shown in Figure 7. The input signals x^ 
through x^ are applied as one set of inputs to a CCD device.- This 
device has a second input similar to the first to which the coefficients 
h through h are applied. After all values have been appropriately 
clocked into the CCD delay line, the resulting output is precisely the 
sum given by Eq. (4). This output may be termed the Vector-Inner Product 
(VIP) . Devices that perform part or all of this computation have been 
described [13, 14, 15 ]. ...The output of any device employed as a VIP can 
be squared in a multiplier and summed to form the quadratic term in the 
exponent of the Gaussian density function (Eq. (1)). This squaring and 
summing can also be performed with a VIP device with slight circuit 


modification. 
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FIGURE 6. TAPPED CHARGE-COUPLED DEVICE DELAY LINE CORRELATOR 
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The quadratic terms computed for each object class are compared in a 
pair wise basis between object classes. This comparison is done in an 
analog comparator and the minimum value determines the class into which 

the signal vector is to be classified. 

The devices which perform the required VIP, accept the signal, and 
a set of stored parameters which are derived from the coefficients 
(statistics) of each density function. These parameters are supplied 
via a Digital-to-Analog Converter from a parameter store. At present 
the only suitable storage mode appears to be binary through employing 
either CCD devices or some other semiconductor technology. Figures 8 a 
and 8b show the implementation of the quadratic classifier. 

4.2 LINEAR CLASSIFIERS 

The linear decision classifier performs the classification process 
in a manner quite similar to the quadratic rule.. However, instead of 
performing a quadratic calculation involving the sum of squares described 
above, a calculation of the form 

(X - M^C <1 (5) 

is performed. This again is basically the capability of the VIP CCD 

device described above. The color space is partitioned into two sections 

consisting of classes H and H, which have mean vectors M and M. 

o 1 o 1 

respectively. For the case where Eq. (5) holds true, the data vector, C. 
x is classified as belonging to class H^, otherwise, it is classified 
as belonging to class H^. The decision is determined by the. vector. 

A method for determining this vector based on using Gaussian distribu- 
tion to describe classes and is given by Richardson and Crane [16] . 
Figure 9 indicates a structure using what has been termed 

i 

the "best linear" classifier which has been shown to provide results., 
equivalent to the quadratic classifier over- a number of tests [16]. 

This procedure employs the CCD VIP device in a more efficient manner, 
requiring, as is shown, only a single device per pair of classes. If 
all ordered pairs are calculated for k classes, — ^ — — CCD VIP devices 
would be required. However, if the decisions are made sequentially. 
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R = qR Q +(l-q)R 1 (o<q<l) 
C ol= 


(For Assumed Equal Covariance Matrices: x replaces x-p^ (new x) 

-2R q V replaces 

where R q is a pooled covariance matrix 

formed from an average of all covariance 
matrices) 


Decisions may be ordered as: 
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C.. terms with ordered access to these terms 
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Fig. 9. Block Diagram of the "Best Linear" Classifier 
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only k-1 such devices are needed.. To be able to perform this operation 

k(k-l) 

serially requires storage of — ^ terms which are then inserted into 

each sum of products device as the decision sequence. 

Finally, if the assumption be made that all classes have the same 
covariance matrix a simpler structure results as shown in Figure 9. 

When tested, performance of this rule did not appear adequate, so no 
further mention is made of it. 

Thus, it appears that linear procedures could provide a simple 
means of implementing a CCD classifier for a spacecraft. The practi- 
cality of this approach, however, may be degraded by the requirement for 
interactive intervention by a ground-based operator. For this reason, 
a clustering algorithm will be discussed. 

4.3 CLUSTERING PROCESSING 

The use of a clustering algorithm for classification has two main 
advantages over more conventional processing: (1) the normally s.eparate 

steps of signature extraction and classification are combined into one 
step, saving time and money, and (2) clustering adapts the signatures 
to variations in the scene, potentially allowing entire large areas to 
be classified with only one set of signatures. 

The clustering procedure, since it is relatively insensitive to 
calibration errors and scene change, and minimizes operator intervention, 
promises to be a more useful technique for a spacecraft classifier. The 
procedure, briefly, is to examine each incoming or stored spectral pixel 
(or vector) and assign it to a cluster based only on the mean and vari- 
ance of all the cluster classes. After all elements have been assigned 
a cluster class or after all clusters have been defined, each cluster 
class code is transmitted in sequence to provide a data reduction equal 
to the encoding needed to define the number of clusters. This would, 
in practice, amount to about twice the number of trained classes, e.g., 
instead of 32 classes, about 64 clusters would occur. This could be ar- 
bitrarily reduced, however, by coding clusters with low-membership into 
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a common or null class. Experiments based on this procedure have given 
good indication that results equivalent to trained classification may 
be had from this method.. ' ;• ‘ " ■ 

• The procedure requires division' to - obtain mean arid variance -pa- 
rameters, as currently -mechanized. .-However, binary -shift approximations 
to division (see Figure 10) may- be employed with loss in accuracy. The 
algorithm used is indicated in Figure 10, and the block diagram of the 
mechanization is shown in Figure 11. 
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Algorithm requires: 


Distance: dfx.NL) = 


N o 

£ (X, -M..) 4 


n 1/2 


j=i 


J LL 


o 2 

i] 


K. 


Mean 


M.. = 1 S x (or, iteratively,) = M. . , 1 (x.-M. ,) 

. 1 K. 1=1 1,1 K. 1 1 ■ 

.1 l 

2 2 2 2 2 2 

Variance: a' = 1. £ (x .-M.) (iteratively) = a -(a -x.+M ) 

1 1“1 ^ x~ x i~ X 1 1 


Binary Approximation 

to Division: 1C replaced by 2 P , where 2 P < IC < 2 P+ 

Rule: d(x,M I .) < T, assign x to class K 

1 K 

d(x 1( M K ) > s, create a new cell 

at each assignment, calculate new mean, variance 

Mechanization: 

Employ one distance function calculator/cluster 
Assign each calculator in turn as x. are tested 

Provide adequate parallelism to obtain desired compression 


Fig. 10. Clustering Algorithm 
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5 

STUDIES OR DEVELOPMENT REQUIRED 

The status of this investigation at this time is such that there 
are several lines of development which should be pursued to assure or- 
derly progress toward the design of a working prototype of a spacecraft 
processor. These may be broken down into five areas for the sake of dis- 
cussion: 1) System Analysis, 2) Pre-processing, 3) Clustering, 4) Spatial 

Pattern Processing and 5) Hardware. Classification is not included be- 
cause adequate research and development exist to assure the capability 
needed for such a processor. 

5.1 SYSTEM ANALYSIS 

The spacecraft processor, as noted above, will exist in one. or more 
spacecraft or aircraft systems which will be configured for various kinds 
of missions in various stages of the development process as time goes on. 
Each mission will require different forms of data to be transmitted to 
various ground systems from different kinds of orbits or flight paths. 

The processing functions required for these missions will vary according 
to these different conditions and with the need for pre-transmission data 
compression. 

A's a result there is a need to determine a matrix, of specifications 
for the series of spacecraft or aircraft and their missions as a first 
step in design. From this matrix, then, it becomes possible to choose 
a small set of typical missions for more detailed analysis. . Finally, 
from the detailed analysis of requirements and specifications, the design 
of a processor able to be applied widely may be derived. . . 

At this time, there are block designs available for many of the basic 
building blocks of such systems: calibration techniques, techniques for. 

dimension reduction, for ratioing, for self-training cluster classifiers 
and for trainable classifiers (MIDAS), and. some approaches toward more 
efficient pre-transmission data compressors. ' 
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A preliminary analysis indicates the utility of geometric correction 
of a relative and geodetic form and even a pattern-matching technique for 
accomplishing this. However, more work is needed in this area. 

5.2 PRE-PROCESSING 

In effect, this area is the most critical for obtaining accurate' 
results by class coding. Classification or recognition .processes are 
rather well defined statistical procedures which will, when applied, yield 
a known accuracy of output for a specified input. The preparation of in- 
put. data, for classification, using procedures often termed "feature, extrac- 
tion", has. a profound influence on .the accuracy . of the. output.. 

Techniques for doing this processing have been commonly called, "pre- . 
processing" techniques. These may be subdivided into the categories of 
calibration (instrumental compensation), haze/scatter/ illumination . . 
correction (situation/ environment compensation), and seasonal or maturity 
correction (natural or temporal, compensation) ... These techniques, are all 
under more or less intensive study and development at the present time, 
but there are areas which need to be brought into sharper focus to pro- - ■ 
ceed toward development of a spacecraft processor ,. since present efforts'- 
aim at ground-based operations and incorporate the capabilities of a 
human interpreter in various ways. In particular, automatic image (data) 
quality analysis needs ..development to enable decisions to transmit data 
or not. Natural or temporal compensation needs study to define, the sets 
of transforms needed to classify efficiently as functions of season, lati- 
tude and local environment. <* ,-*■ • 

Hardware research and development is. required for implementation of 
these techniques. This is the case for the- Vector Inner Product devices, 
which need refinement and single-chip mechanization. . It is also.: the case 
for division by wide-band signal variables to obtain ratioed spectral 
functions. And, since control of operation requires, . and wilT probably 
always require,, some flexibility in operations, there is a need for 
development of properly qualified high-speed, microprocessors . r- ^ 
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5 . 3 CLUSTERING 

Since classification by clustering or cluster classification is the 
technique of preference in non-trainable, non- interactive procedures, 
insofar as it greatly improves throughput, some more concentrated study 
of this area is needed to verify the accuracy of the technique in as 
wide a variety of applications as may be foreseen for the processor sys- 
tem, Development is needed to obtain more adaptivity in the technique 
and to assess the adequacy of that adaptivity already inherent in present 
procedures both in continuous adaptation as the scan progresses through 
the scene and in discontinuous adaptation as the scene is obscured or not 
scanned. Study of the inclusion or exclusion of covariance information 
in the cluster procedure as a function of the adequacy of pre-processing 
is also required. The trade-off here is the complexity of the pre-processor 
versus the complexity of the cluster classifier, since covariance reduc- 
tion may be. done in the preprocessor more efficiently. 

Hardware developments are also required in this area. The present 
cluster procedure requires simultaneous computation of the VIP distance 
measure,, and is already mentioned above. Here the VIP elemental-block 
function itself is the basic unit. There is a requirement for a fast 

-7 3 4 

and small (~ 10 sec and 10 to 10 bit) memory for cluster parameter 

processing, with a fast microprocessor for parameter update. There is 

8 9 

also a requirement for a bulk (10 to 10 bit) memory of medium-fast 
speed (~ 10 ^ sec) which can hold the image data or its coded equivalent. 
These seem to be receiving adequate attention in the development of fast 
semiconductor RAM and bubble memories. However, we are not aware of any 
development work aimed at developing a microprocessor for arithmetic 
operations and RAM accessing, suitable to meet the specific spacecraft 
requirements of high-speed, high-reliability, and relatively low power 
while being relatively immune to the ambient radiation levels of the 
orbital environment. 

5.4 SPATIAL PATTERN PROCESSING 

There are two areas of spatial processing in which requirements exist 
for a spacecraft processor: 1) spatially coherent clustering and 2) scene 

point identification. These are aimed, respectively, at the improvement 
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of cluster resolution or identification arid the location of geodetic 
reference points for geometric correction of imaged data. Both areas are 
under development sponsored by the NASA-SRT and military programs but, : 
again, need specific orientation to a spacecraft processor. ' 

5.4.1 Spectro-Spatial Clustering 

Spatially coherent clustering (also called spectro-spatial clus- 
tering or "blob" clustering) adds to conventional multivariate or multi- 
spectral clustering, the spatial dimensions of the scene points so that 
clustering may be done on objects with spatial contiguity, such as fields, 
lakes or forests. The number of clusters is thereby increased, leading 
to a reduction in compression, but allows the identification of fields, 
lakes, etc, as unique objects or assists in separation of spatially dis- 
tinct objects when this is useful. Such may be the case, when one would' 
like to know, for example, how many lakes of a certain set of size distri-- 
butions are in a scene, or how many crop fields of certain sizes and crop 
classes are in the scene. r ! *' 

If this information is sufficient, then a greater compression may 
be had in transmission. In such a case, rather than transmit the 
identity of every point in a scene, the mean x-y coordinates of the 
spatial cluster may be transmitted along with the size arid identity of 
the object. Greater accuracy in identification may be obtained iri this 
form of processing since the inter-cluster distance is increased by the 
addition of spatial dimensions.' J 

The impact of this technique on a spacecraft processor is twofold. 
First, the number of clusters is increased, requiring a larger number 
of Vector Inner Product distance elements (one for each cluster) and some 
increase in the size of the fast memory for the additional cluster 'and 
their parameters. However, no greater speed is required. ' ‘ r 

Second, the rate of transmission may be increased or decreased over 
that possible with clustering of spectral samples. If .all cluster 
codes are transmitted on a point-by-point basis, then, since the code 
is longer to describe the larger number of classes, the data is less 
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compressed for transmission. If, on the other, only the parameters of 
all cluster classes are transmitted, that is to say, the means, variances, 
occupancy and spatial coordinate centers, then an additional and sizeable 
reduction, possibly on the order of 10 to 100 may be obtained. 

It should be noted that all these methods can be used with the same 
basic hardware and that a specific spacecraft processor may be set up to 
provide all three processes, switchable as needed by command from the 
control station. 

5.4.2 Cellular Automata Pattern Processing 

Size, weight, cost and processing speed requirements dictate that 
data processing approaches be taken that differ radically from the more 
common digital computer pattern recognition techniques. Two generic 
concepts have been considered; Optical Fourier Transform Matched 
Filtering and Image Analyzing Cellular Automata. As a result of a 
preliminary evaluation, the Optical Matched filter approach was de- 
emphasized in favor of the Cellular Automata concept. 

Basically, the use of the cellular automata for pattern recognition 
centers around the concept of the neighborhood transform [18, 19] 
illustrated in Figure 12. A transform T is applied globally to a recn 
tangular array A to produce a resultant array B. The element b^. of the 
array B is derived from the local neighborhood of the element a_^. of A, 
typically three-by-three. Additionally, the neighborhood function might 
also include row and column indices i and j as arguments to the neigh- 
borhood function. Transformation hardware complexity is significantly 
reduced by initially considering only binary arrays and binary transforms. 
The basic hardware structure for implementing the binary neighborhood 
transform is shown in Figure 13. 

As the elements of array A are sequentially scanned they are fed into 
a linear array of shift register stages, preferably charge-coupled de- 
vices. Nine shift register stage outputs act as the binary address 
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FIGURE 12. NEIGHBORHOOD TRANSFORM CONCEPT 














Figure 13. NEIGHBORHOOD TRANSFORM PROCESSING HARDWARE 


37 


































FORMERLY WILLOW RUN LABORATORIES. tHE UNIVERSITY OF MICHIGAN 


inputs c .... c 0 of a 2" bit random access memory (RAM) which stores 
o o 

the Boolean transform function. Address bits c^ and c^^ reflect the even/ 
odd parity of the row and column of the currently scanned element. Since 
the scanner, CCD shift registers and RAM operate from a common clock/ 
the RAM output occurs at the same rate as the scanned input. Neighbor-, 
hood Transformation Processing Stages, each consisting of CCD and RAM 
elements, may then be cascaded to implement more complex array transforms. 

Research in this area is key to the development of on-board pro- 
cessing techniques which are reliable enough to qualify for operational 
use in a spacecraft environment and sufficiently sophisticated to. pro- 
vide the reduction in ground data processing required for economical 
use of remote sensing on a global basis. 

5.4.3 Spatial Correlation 

The use of multitemporal data sets for crop identification was de- 
scribed in Section 3.1. This use of multiple data sets implies that 
some of the data to be registered must be stored over a considerable 
length of time and it also implies that suitable mass storage is avail- 
able. At the present time it is not certain that storage and registry 
of data on board a spacecraft is a viable technique for the purpose of 
reducing the amount of data to be transmitted on the downlink. However, 
a brief study should be made to include the latest projections of satel- 
lite storage capabilities along with recent spatial registry techniques 
to see if there are any advantages to be gained in the overall classifi- 
cation process. f 
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6 

CONCLUSIONS AND RECOMMENDATIONS 


The introduction of modern sensors aboard spacecraft for earth re- 
sources survey permit large portions of the earth to be rapidly remotely 
sensed by these devices. Since r jthese','muitispectral scanners have fine 
spectral and spatial resolution, the amount of data collected will have 
typical average data rates . from-10 to ^TOCT megabit's/sec. Although a 
special purpose processing' system called MIDAS* has been developed as an 
attempt to cope with these':;rates -,and overcome; the bottlenecks encountered 

*V. .‘3 ^ *• • • : ^ * • ; ■ 

with ground processing techniques employing general purpose computors, 
a special purpose ground-based system does not reduce the problems asso- 
ciated with wide band data transmission links or storage of these trans- 
mitted data. There is a need for processing scanner data aboard a 
spacecraft as a means of reducing the peak and average loads of data 
to be transmitted to the ground, the data storage requirements on~the 
ground, and also the number, size, and cost of processing systems on 
the ground. 

The development of charge-coupled devices makes it now feasible 
to address the spacecraft on-board processing of remotely sensed data. 

It is recommended that further work be done in this area. The specific 
functions which appear as most amenable to this approach are: Pre- 

processing, classification, and clustering. These operations could be 
performed prior to other, standard, data compression techniques which 
are used to reduce the data load on the downlink system. 


* Multivariate Interactive Digital Analysis System 
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